Skip to content

fix signed overflow in calculate() and MathLib constant folding - #9030

Open
Nussu06 wants to merge 2 commits into
cppcheck-opensource:mainfrom
Nussu06:constant-folding-signed-overflow
Open

Nussu06 wants to merge 2 commits into
cppcheck-opensource:mainfrom
Nussu06:constant-folding-signed-overflow

Conversation

@Nussu06

@Nussu06 Nussu06 commented Nov 14, 2026

Copy link
Copy Markdown
Contributor

calculate() in lib/calculate.h, MathLib::add/subtract/multiply/mod and valueFlowEnumValue fold integer constants from the checked code in MathLib::bigint with plain signed arithmetic, so code like x = 0x7fffffffffffffff + 1;, A<0x8000000000000000 % -1> or enum E { A = 0x7FFFFFFFFFFFFFFE, B, C }; makes cppcheck itself hit signed integer overflow (UBSan reports in calculate.h, mathlib.cpp and valueflow.cpp, and the % -1 case can trap on x86-64). Added a calculateArithmetic helper that does +, - and * for signed integer types on the unsigned type and used it at those sites, and x % -1 now returns 0, so folded values are the same as before and only the undefined behaviour is gone. With this, testrunner is clean under ASan+UBSan with signed-integer-overflow enabled (cmake/dynamic_analyzer_options.cmake still disables it with a TODO); I added asserts to TestMathLib::calculate and TestValueFlow::valueFlowCalculations plus a new TestSymbolDatabase::enum21, and they fail under UBSan without the fix. There is no trac ticket for this yet, could you create one? I will update the title to "Fix #NNNNN (...)" once it exists.

@dmcppcheck

Copy link
Copy Markdown
Contributor

There should probably be a trac ticket for this PR. Please create it, or ask us to create it. Then the PR title should be updated.

@dmcppcheck dmcppcheck left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI review

I found no problems in the changed code. Under a -fsanitize=signed-integer-overflow build, all the cases the PR names no longer overflow, and the folded values stay the same as before (they wrap). One related site is still left: ValueFlowAnalyzer::writeValue in lib/vf_analyzers.cpp:374 does value->intvalue += (inc ? 1 : -1); for ++/--. So long long x = 0x7fffffffffffffff; x++; still makes cppcheck overflow a signed integer. The PR does not touch this path, and its tests don't cover it, so the claim that it gets rid of this kind of undefined behaviour is not complete. The same calculateArithmetic helper could be used there too.

Testing

  • I built cppcheck and testrunner from pr/ with the normal Makefile. ./testrunner TestMathLib TestValueFlow TestSymbolDatabase: 555 tests, 0 failed.
  • I built a second copy of pr/ with CXXFLAGS="-O1 -g -fsanitize=signed-integer-overflow,undefined -fno-sanitize-recover=signed-integer-overflow" and ran it on small inputs, one line per file:
    • 0x7fffffffffffffff + 1, 0x7fffffffffffffff * 2, LLONG_MIN % -1 in an expression and as a template argument, enum E { A = 0x7FFFFFFFFFFFFFFF, B, C };, x += 1 / y *= 3 after x = LLONG_MAX, unary minus and / -1 of LLONG_MIN, a condition x == LLONG_MAX followed by x + 1, and a loop near LLONG_MAX: no UBSan reports.
    • With --debug, the folded values are as expected. + gives -9223372036854775808, * gives -2, and the enum gives B = LLONG_MIN and C = LLONG_MIN+1. (LLONG_MIN <=> 1) still folds to -1.
  • A case that still overflows (not part of the PR's changes):
    // sp/s9.cpp
    void k() { long long x = 0x7fffffffffffffff; x++; ++x; }
    /work/pr_ubsan/cppcheck --std=c++20 --enable=all -q sp/s9.cpp
    lib/vf_analyzers.cpp:374:29: runtime error: signed integer overflow: 9223372036854775807 + 1 cannot be represented in type 'long long int'
        #0 ValueFlowAnalyzer::writeValue(...) lib/vf_analyzers.cpp:374
        #1 ValueFlowAnalyzer::update(...) lib/vf_analyzers.cpp:837
    
  • I did not run the full testrunner under ASan+UBSan, did not test the Boost int128_t build, and did not build base under UBSan.

Automated review of 8c80299 by Claude. The pull request was built and tested locally.

@Nussu06

Nussu06 commented Nov 15, 2026

Copy link
Copy Markdown
Contributor Author

Good catch on ValueFlowAnalyzer::writeValue. The ++/-- path in lib/vf_analyzers.cpp now goes through calculateArithmetic as well, so long long x = 0x7fffffffffffffff; x++; wraps to LLONG_MIN like before without the signed overflow. I added a case for it in TestValueFlow::valueFlowAfterAssign.

About the trac ticket: could you create one for this? I'll change the title to "Fix #NNNNN (...)" as soon as it exists.

@dmcppcheck

Copy link
Copy Markdown
Contributor

This pull request has merge conflicts that must be resolved.

@dmcppcheck

Copy link
Copy Markdown
Contributor

Test results for commit fd6eddb (tools/test-my-pr.py, main compared to this PR):

Test: http://ec2-16-170-140-253.eu-north-1.compute.amazonaws.com/pr-9030/
packages: 1000
Differing warnings: 0 (+0 -0)
Timing: -0.0% (http://ec2-16-170-140-253.eu-north-1.compute.amazonaws.com/pr-9030/2026-11-15_16-33-57-fd6eddb9553e_timing.html)

Posted automatically by the cppcheck PR test runner. +N: warnings only with this PR, -N: warnings only with main. The AI review is written by Claude and can be wrong.

@Nussu06
Nussu06 force-pushed the constant-folding-signed-overflow branch from fd6eddb to 14439b1 Compare November 17, 2026 19:23
@Nussu06

Nussu06 commented Nov 17, 2026

Copy link
Copy Markdown
Contributor Author

Rebased on main. The conflict was with #9006, which added the compound assignment labels to calculate(). +=, -= and *= now go through calculateArithmetic as well, so x += 1 at LLONG_MAX doesn't overflow either. The full testrunner passes and is clean under -fsanitize=signed-integer-overflow.

I still don't see a trac ticket for this. Could one of you create it? I'll update the title to "Fix #NNNNN (...)" once it exists.

@dmcppcheck dmcppcheck left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI review

I found no new problems in the code the PR changes. The ++/-- site in vf_analyzers.cpp from the earlier review now goes through calculateArithmetic, and long long x = LLONG_MAX; x++; no longer overflows under UBSan. The folded values are the same as before: they wrap.

Two signed overflows are still left in code the PR does not touch, so the claim that the overflow is gone is not complete:

  • Unary minus in lib/programmemory.cpp:2107 (v.intvalue = -v.intvalue;), hit by long long m = LLONG_MIN; long long n = -m;.
  • #if arithmetic in externals/simplecpp/simplecpp.cpp:1298, hit by #if 0x7fffffffffffffff + 1 < 0.

Neither line is in the diff, so I can't comment on them inline.

Comments

# Comment By Where Status
1 PR needs a trac ticket and an updated title @dmcppcheck general ⚠️ Open
2 ++/-- in ValueFlowAnalyzer::writeValue still overflows a signed integer AI review general ✅ Fixed

@Nussu06, please fix, resolve or reply to the open and new comments: #1.

Testing

  • Normal build: I built cppcheck and testrunner in pr/ with the normal Makefile.

    • ./testrunner TestMathLib TestValueFlow TestSymbolDatabase: 555 tests, 0 failed.
    • Full ./testrunner -q: 5420 tests, 0 failed.
  • UBSan build: I built a second copy of pr/ with CXXFLAGS="-O1 -g -fsanitize=signed-integer-overflow -fno-sanitize-recover=signed-integer-overflow -std=c++11". I ran it on files that cover:

    • x++ / --x at LLONG_MIN and LLONG_MAX
    • +=, *= and -= past the limits
    • an enum with B = A - 1 and A = LLONG_MIN
    • template arguments LLONG_MAX + 1, LLONG_MIN % -1 and LLONG_MAX * 3
    • <=>, a loop near LLONG_MAX, and conditions a == LLONG_MAX followed by a + 1 / a * 2
    • reverse analysis (x++; if (x == LLONG_MIN))
    • unsigned long long wrapping and int overflow

    None of the code the PR changes gave a UBSan report. The existing integerOverflow warning for A - 1 in the enum still comes.

  • Remaining overflows outside the diff (both are in code the PR does not change):

    // scratch/u2.cpp
    void h() {
      long long m = -0x7fffffffffffffff - 1;
      long long n = -m;
    }

    /work/pr_ub/cppcheck -q u2.cpp

    lib/programmemory.cpp:2107:38: runtime error: negation of -9223372036854775808 cannot be represented in type 'long long int'; cast to an unsigned type to negate this value to itself
    
    // scratch/t5.c
    #if 0x7fffffffffffffff + 1 < 0
    int a;
    #endif

    /work/pr_ub/cppcheck -q --enable=all t5.c

    externals/simplecpp/simplecpp.cpp:1298:20: runtime error: signed integer overflow: 9223372036854775807 + 1 cannot be represented in type 'long long int'
    
  • Not done: I did not build testrunner under UBSan, did not test the Boost int128 configuration, and did not build base/.


Automated review of 14439b1 by Claude. The pull request was built and tested locally.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants